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The  problem  of  stabilization  and  control  of  nonlinear  stochastic 
systems  observed  by  noisy  measurement  data  arises  in  many  Air 
Force  system  applications.  Inherent  in  this  problem  is  the  problem 
of  processing  noise-contaminated  measurement  data  to  obtain  accurate 
estimates  of  the  state  of  the  system.  If  it  is  possible  to  estimate  the 
state  of  the  system  accurately,  then  well-known  classical  deterministic 
control  techniques  may  often  be  used  to  give  adequate  system  performance. 
This  approach  will  greatly  reduce  the  complexity  of  the  control  algorithm 
over  that  required  by  a truly  "optimal"  stochastic  control  policy.  On 
the  other  hand,  the  use  of  recently  developed  filtering  techniques  in 
place  of  the  simpler  linearized  or  extended  Kalman  filters  can  greatly 
increase  the  accuracy  of  the  state  estimates  ana;  thereby,  improve 
system  performance  and  alleviate  divergence  problems. 

A straightforward  approach  to  the  stochastic  control  problem 
would  be  to  use  the  recent  research  results  giving  approximate  a 
posteriori  densities  in  conjunction  with  the  principle  of  optimality  in 
order  to  solve  the  optimal  control  problem  using  the  well  known  stochastic 
dynamic  programming  equations.  Preliminary  studies  of  this  approach 
show  two  things.  First,  even  for  very  simple  systems  in  which  it  is 
possible  to  obtain  tractable  approximate  densities,  it  is  very  difficult 
to  obtain  solutions  for  even  approximate  controls  in  this  manner. 

Secondly,  even  though  the  control  algorithms  developed  can  in  no  way 
be  considered  practical  because  of  the  extensive  calculations  involved, 
they  can  lead  to  greatly  reduced  cost  even  for  very  simple  problems. 

This  second  factor  does  make  the  continued  investigation  of  the  sto- 
chastic control  problem  interesting,  but  the  first  indicates  another 
approach  should  be  used. 

The  approach  taken  in  this  contract  has  been  to  investigate 
the  effect  of  already  developed  nonlinear  filters  in  the  feedback 


loops  of  nonlinear  stochastic  control  systems  to  be  followed  by  optimal 
or  nearly  optimal  deterministic  controls.  This  in  effect  can  force 
the  separate  structure  of  the  linear  quadratic  Gaussian  problem  on  the 
more  general  nonlinear  stochastic  control  problem,  but  will  make  use 
of  the  latest  developments  in  the  field  on  nonlinear  estimation.  In 
particular,  nonlinear  measurements  of  basically  linear  systems  will 
be  investigated  since  the  cost  of  transducers  often  increases  markedly 
from  the  linearity  constraint.  It  may  be  that  the  cost  of  a small  computer 
in  a control  loop  could  be  completely  saved  by  a reduction  in  the  cost 
of  the  transducers  required. 

Work  on  this  contract  has  allowed  continued  investigation  into 
the  development  and  use  of  nonlinear  filters  in  conjunction  with  deter- 
ministic control  laws.  Much  of  the  detail  on  the  work  performed 
under  Contract  AFOSR -F44620- 75-C- 0023  is  contained  in  the  publica- 
tions listed  below,  which  have  been  published,  accepted  or  submitted 
since  the  beginning  of  this  contract. 


Alspach  , D.  L.  , "A  Discussion  of  the  Relationships  Between 
the  Dual  Goals  of  Stochastic  Control,  " invited  chapter  in  forth- 
coming book  Nonlinear  Estimation  and  Filtering  Theory:  A 
Status  Review,  edited  by  E.  Stear  to  be  published  by 
Marcel  Dekker,  in  preparation. 

Alspach,  D.  L.  , "A  Discussion  of  the  Relationships  Between 
the  Dual  Goals  of  Stochastic  Control,  " An  International  Journal 
of  Computers  and  Electrical  Engineering,  Vol.  4,  No.  1, 
January  1977. 

Alspach,  D.  L.  , "A  Stochastic  Regulator  Using  a Certainty 
Equivalence  Control  with  a Nonlinear  Filter  for  Processing 
Hard-Limited  Data,  " Accepted  for  publication  in  Information 
Sciences. 

Scharf,  L.  L.  and  D.  L.  Alspach,  "Nonlinear  State  Estimation 
in  Observation  Noise  of  Unknown  Covariance,  " Proceedings 
of  the  1976  Joint  Automatic  Control  Conference  (West  La 
Fayette,  Indiana,  July  27-30,  1976).  Also  accepted  for 
publication  in  the  International  Journal  of  Control. 


(5) 


i 


i 

■; 

I 

e 

. 

'1 

> 

« 


* 

f; 


i 

L 


Alspach,  D.  L.  , "Nonlinear  Filters  in  Feedback  Control,  " 
Proceedings  of  the  Sixth  Symposium  on  Nonlinear  Estimation 
Theory  and  Its  Applications,  San  Diego,  California  (Sept.  1975i. 

(6)  Alspach,  D.  L.  , "A  Certainty  Equivalence  Control  wit!  a Non- 
linear Filter  in  the  Feedback  Loop,  " Proceedings  of  the  1 Cj 7 5 
IEEE  Symposium  on  Decision  and  Control,  Houston,  Texas 
(December  10-12,  1975). 

(7)  Alspach,  D.  L.  , "A  Stochastic  Control  Algorithm  for  Systems 
with  Control  Dependent  Plant  and  Measurement  Noise,  " An 
International  Journal  of  Computers  and  Electrical  Engineering, 
Vol.  2,  No.  4,  November  1975. 

(8)  Alspach,  D.  L.  , "A  Gaussian  Sum  Approach  to  the  Multitarget 
Identification-Tracking  Problem,  " Automatica,  Vol.  11, 

pp.  285-296  (August  1975). 

Only  a brief  outline  of  the  work  contained  in  the  above 
publications  is  given  here.  A summary /review  paper  is  being 
prepared  as  an  invited  chapter  of  a text  on  nonlinear  estimation, 
edited  by  Dr.  E.  Stear.  The  paper  is  entitled  "Approximate 
Solutions  of  the  Nonlinear  Filtering  Equations"  and  the  book  will  be 
entitled  "Nonlinear  Estimation  and  Filtering  Theory:  A Status  Review.  " 
The  work  done  on  the  contract  to  date  including  that  in  the  publications 
above  will  be  summarized  in  some  detail  in  this  review  chapter, 
as  well  as  work  by  other  workers  in  the  field. 

A general  philosophical  approach  to  stochastic  control  is 
discussed  in  (2)  above.  The  method  of  aligning  the  "dual"  goals  of 
the  general  optimal  stochastic  control  as  a design  tool  is  discussed. 
When  the  two  goals  are  exactly  aligned,  the  certainty  equivalence 
control  is  optimal  and  no  additional  intentional  "probing"  is  required. 

If  these  goals  are  "anti-aligned,  " demanding  opposing  controls,  the 
certainty  equivalence  control  can  be,  locally  at  least,  the  worst 
control  possible.  An  example  with  this  characteristic  has  been  dis- 
cussed in  publication  (2). 
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In  publication  (3)  we  consider  a simple  example  of  a nonlinear 
filter  in  a feedback  loop.  For  this  case  the  ,!dual  goals'  are  aligned 
and  it  is  shown  that  the  performance  of  the  system  is  very  close  to  that 
of  an  optimal  stochastic  control.  Since  the  optimal  stochastic  control 
algorithm  is  very  difficult  to  calculate,  this  is  done  by  comparing  the 
performance  to  a known  lower  bound.  This  lower  bound  is  found  in 


the  following  manner.  It  is  clear  that  there  is  more  information  about 
the  state  in  a linear  measurement  of  the  state  contaminated  by  noise 
z Tlv(k)  than  in  the  hard-limited  version  y (k): 

■ 'i-iN  rij_, 

ZLIN(k)  = Hk\  + Vk 

y (k)  = SIGN(z  (k))  = SIGN(H  x + v ) . 
rf-L  FIN  k k k 

With  the  linear  measurement  function,  the  optimal  stochastic  control 
is  given  by  the  separation  theorem.  It  is  clear  that  the  performance 
of  the  optimal  stochastic  control  system  with  only  the  hard-limited 
function  available  will  be  worse  than  the  system  with  the 

linear  function  z^j.^.(k).  It  is  shown  in  publication  (3)  that  given  only 
yHlJk)’  Perf°rmance  of.  the  certainty  equivalence  control  with 
a nonlinear  filter  is  close  to  that  of  the  optimal  control  system  with 
the  better  linear  measurement. 

In  publication  (4),  a new  nonlinear  filter  was  developed  in  con- 
junction with  Dr.  L.  Scharf  of  Colorado  State  University.  This  paper 
considers  the  adaptive  Kalman  filtering  problem  where  only  the 
measurement  noise  covariance  is  unknown.  A new'  parallel  filter- 
ing algorithm  is  developed.  It  is  one  of  several  nonlinear  filters  that 
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will  be  used  in  conjunction  with  certainty  eauivalence  controls  in  the 
next  phase  of  this  study. 


In  publication  (5),  the  effect  of  control-dependent  plant  and 
measurement  noise  on  the  feedback  control  is  discussed.  It  is 
shown  that  the  goals  are  effectively  aligned,  but  not  of  the  same 
magnitude.  The  use  of  any  control  action  reduces  the  accuracy  of 
the  state  estimates.  Thus,  the  effect  of  such  control -dependent 
noise  is  to  induce  "caution1'  on  the  control. 

In  publication  (6),  the  filter  for  processing  hard-limited 
measurement  data  was  first  introduced.  In  Publication  (7),  a 
filter  for  state  estimation  in  systems  with  state  and  control-dependent 
measurement  noise  was  introduced.  In  publication  (8),  Gaussian  sum 
filters  are  used  in  a Bayesian  approach  to  the  multitarget  tracking 
problem. 
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